1. Field of the Invention
The present invention relates an image taking apparatus for acquiring image information using an image taking element, and in particular to a program created to output a perspectively projected image obtained by performing a image shift correction process for an acquired fisheye image, or an image taking apparatus mounted with the program and configured to output the perspectively projected image.
2. Related Background Art
Wide-viewing-angle image taking by a fisheye optical system is employed in various fields such as monitoring, remote operation, TV conference and endoscope. The wide-viewing-angle image taking is characterized in capable of projecting a super-wide-viewing-angle image with an field angle of more than 180° onto a limited area of the light receiving surface of an image taking element due to its special projection method and capable of acquiring even such a super-wide-viewing-angle image that theoretically could not be acquired by a common image taking apparatus for taking a perspectively projected image. However, in the projection method specific to fisheye optical system, the form of a taken image is distorted and is significantly different from the form of an original object.
As an approach for solving this problem, there is proposed an apparatus for converting an acquired fisheye image to a perspectively projected image and displaying the converted image on a display in U.S. Pat. No. 5,185,667, U.S. Pat. No. 5,313,306, U.S. Pat. No. 5,359,363, U.S. Pat. No. 5,384,588, U.S. Pat. No. 5,764,276, U.S. Pat. No. 5,877,801, U.S. Pat. No. 5,903,319, U.S. Pat. No. 5,990,941, U.S. Pat. No. 6,002,430, US AA2002097332, U.S. Pat. No. 6,201,574, U.S. Pat. No. 6,243,131, U.S. Pat. No. 6,256,061, U.S. Pat. No. 6,301,447, U.S. Pat. No. 6,603,502 and U.S. RE36207 (hereinafter collectively referred to as “U.S. Pat. No. 5,185,667 and the like”). In U.S. Pat. No. 5,185,667 and the like, the pan, tilt, rotation and magnification are specified for any image area in an image in a hemispherical field of view which has been acquired by fisheye image taking, and coordinate transformation is performed for the image area with the use of X-map and Y-map of two-dimensional mapping to enable omnidirectional observation of the image.
In WO98/27718A1 (specifically in FIGS. 1 to 3), there is proposed electronic zoom using image information density distribution of an image taken by a fisheye optical system. Specifically, an image taken by a fisheye optical system has a characteristic that the compression rate is higher in its peripheral part and the magnification ratio is higher in the central part, and by utilizing this characteristic, electronic zoom can be performed without reducing the resolution.
Furthermore, in Japanese Patent Application Laid-Open No. 8-307753 (specifically in FIG. 10), there is proposed an apparatus capable of following the motion of an observer's head and thereby displaying data of a part of a wide-viewing-angle image taken with a fisheye lens without causing distortion. Especially in the third embodiment described in this patent document, a conversion process is performed for displaying an image taken with a fisheye lens employing an equidistant projection method, as a plane without distortion, and then, an image shift correction process for correcting image shift caused by shaking of an image taking apparatus is further performed for the image. The correction process is performed when the amount of pixel displacement corresponding to image shift is equal to or below a predetermined number of pixels (for example, two pixels).
In the apparatus proposed in U.S. Pat. No. 5,185,667 and the like, however, execution to a pan, tilt, rotation or zoom function is set for any area by specifying a viewing angle and a magnification from input means, solving a conversion equation by means of a microcomputer and setting a conversion table for a two-dimensional map. Therefore, there is a problem that, since it is required to solve the conversion equation even for slight change in the viewing angle or the magnification every time such change occurs, a lot of calculation burden is imposed on the microcomputer.
In the apparatus proposed in WO 98/27718A1, by using image information density distribution of a fisheye image, it is possible to acquire an electronically zoom image the resolution of which has not been deteriorated much. However, there is a problem that as the zoom ratio is higher, distortion of the image is caused by vibration given to the apparatus body, such as shaking and resolution deterioration is caused.
In the apparatus proposed in Japanese Patent Application Laid-Open No. 8-307753, the criteria for performing an image shift correction process is as follows. That is, the correction process is controlled to be performed if the detected amount of pixel displacement (the amount of displacement between pixels (positions) on which two images are displayed respectively when the images should be displayed on the same pixel (position) but displayed on the different pixels (positions)) is larger than a predetermined number of pixels when they are compared. However, the value of the amount of pixel displacement due to image picture shift or positional change is significantly influenced by the amount of vibration given to the apparatus body, the zoom ratio and the like. Therefore, the value is determined depending on each of all zoom ratios, and if determination whether or not to perform the correction process is controlled based on a single threshold (the number of pixels), the tolerance value for image shift is different for each zoom ratio. This not preferable.